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IN THE CLAIMS : 

The claims are not further amended herein but are presented for the Examiner' s 
convenience. 

l.to9. (Cancelled). 

10. (Previously Presented) An image processing apparatus for composting an 
image of a virtual object and an image of a physical space to generate a mixed reality image and 
causing a HMD to display the mixed reality image, comprising: 

a database which holds data used for generating the image of the virtual object; 

an image capturing unit which is attached to the HMD and captures the image 
of the physical space; 

a first measurement unit which measures a position and orientation of the 

HMD; 

an object manipulation unit which is used by a user wearing the HMD in order 
to operate a position and orientation of the virtual object; 

a second measurement unit which measures a position and orientation of said 
object manipulation unit; 

an operation panel which can be operated by the user, arranged at a position in 
the physical space within a viewing field of said image capturing unit attached to the HMD, 
displays an operation panel image used for editing the virtual object, and is capable of receiving a 
user instruction of editing the virtual object input by the user; 
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an operation panel image generation unit which generates the operation panel 
image by using the data held in said database, and outputs the generated operation panel image to 
said operation panel; 

a rendering unit which updates the data held in said database according to the 
user instruction received via said operation panel and the measurement result of said second 
measurement unit, and renders, by using the updated data, the image of the virtual object 
according to the measurement results of said first and second measurement units; 

a composition unit which composites the image of the virtual object rendered 
by said rendering unit and the image of the physical space captured by said image capturing unit 
to generate the mixed reality image; and 

a HMD which displays the mixed reality image generated by said composition 

unit. 

11. (Previously Presented) The apparatus according to claim 10, wherein the 
image of the virtual object is generated on the basis of 3D CAD data of the virtual object, and 
said operation panel displays an assembly tree based on the 3D CAD data. 

12. (Previously Presented) The apparatus according to claim 11, wherein a part, 
which is obtained by enlarging a designated part of the assembly tree included in the operation 
panel image, includes a component name contained in the assembly tree. 
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13. (Previously Presented) The apparatus according to claim 10, wherein said 
operation panel includes a display device and an operation device, wherein the display device 
displays the operation panel image, and wherein the operation device is used for inputting the 
user instruction. 

14. (Previously Presented) The apparatus according to claim 10, wherein the 
HMD can present the mixed reality image to two eyes of a user who wears the HMD. 

15. (Previously Presented) An image processing method of composting an image 
of a virtual object and an image of a physical space to generate a mixed reality image and causing 
a HMD to display the mixed reality image, comprising the steps of: 

holding data used for generating the image of the virtual object in a database; 
providing an image capturing unit, which is attached to the HMD, to capture 
the image of the physical space; 

measuring a position and orientation of the HMD with a first measurement 

unit; 

operating an object manipulation unit, by a user wearing the HMD, in order to 
position and orient the virtual object; 

measuring a position and orientation of the object manipulation unit with a 
second measurement unit; 

receiving a user instruction of editing the virtual object, input by the user using 
an operation panel, wherein the operation panel is arranged at a position in the physical space 
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within a viewing field of the image capturing unit attached to the HMD and displays an operation 
panel image used for editing the virtual object; 

generating the operation panel image by using the data held in the database 
with an operation panel image generation unit and outputting the generated operation panel 
image to the operation panel; 

updating the data held in the database according to the user instruction 
received via the operation panel and the measurement result of the second measurement unit, and 
rendering, by using the updated data, the image of the virtual object according to the 
measurement results of the first and second measurement units; 

compositing the rendered image of the virtual object and the captured image of 
the physical space to generate the mixed reality image; and 

displaying the generated mixed reality image on the HMD. 

16. (Previously Presented) A computer-readable storage medium encoded with a 
computer program for an image processing method of composting an image of a virtual object 
and an image of a physical space to generate a mixed reality image and causing a HMD to 
display the mixed reality image, comprising the steps of: 

holding data used for generating the image of the virtual object in a database; 

providing an image capturing unit, which is attached to the HMD, to capture 
the image of the physical space; 

measuring a position and orientation of the HMD with a first measurement 

unit; 
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operating an object manipulation unit, by a user wearing the HMD, in order to 
position and orient the virtual object; 

measuring a position and orientation of the object manipulation unit with a 
second measurement unit; 

receiving a user instruction of editing the virtual object, input by the user using 
an operation panel, wherein the operation panel is arranged at a position in the physical space 
within a viewing field of the image capturing unit attached to the HMD and displays an operation 
panel image used for editing the virtual object; 

generating the operation panel image by using the data held in the database 
with an operation panel image generation unit and outputting the generated operation panel 
image to the operation panel; 

updating the data held in the database according to the user instruction received 
via the operation panel and the measurement result of the second measurement unit, and 
rendering, by using the updated data, the image of the virtual object according to the 
measurement results of the first and second measurement units; 

compositing the rendered image of the virtual object and the captured image of 
the physical space to generate the mixed reality image; and 

displaying the generated mixed reality image on the HMD. 
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